PME-NA 2011 Proceedings 284 


IMPULSIVE-ANALYTIC DISPOSITION: INSTRUMENT PILOT-TESTING 


Kien Lim Osvaldo F. Morera 
University of Texas at El Paso University of Texas at El Paso 
kienlim@utep.edu omorera@utep.edu 


The likelihood-to-act (LtA) survey measures impulsive and analytic dispositions in solving 
mathematics problems. The current version has 16 impulsive and 16 analytic items. Its validity 
was assessed using a sample of 27 in-service and 92 pre-service teachers. Both the impulsive 
and analytic subscales were found to have internal consistency reliability, but they were not 
correlated with one another. The impulsive subscale was predictive of correctness in classifying 
the LtA items. The analytic subscale was predictive of how well a participant would perform in 
Part 2 of a math test after taking Part I and being warned that some items could be tricky. 


Many pre-service teachers for elementary and middle grades appear to have an impulsive 
disposition—a proclivity to spontaneously proceed with an action that comes to mind without 
analyzing the problem situation and without considering the relevance of the anticipated action 
to the problem situation (Lim, 2006). For example, Lim (2009) found that 22 out of 28 pre- 
service middle-school teachers used the same strategy to solve two superficially-similar but 
structurally-different problems. 

As mathematics educators, we want to help students advance from impulsive disposition to 
analytic disposition—where one analyzes the problem situation (Lim, 2006). An instrument that 
can identify problem-solving disposition will be valuable to both students and teachers. With this 
purpose in mind, we developed the likelihood-to-act survey (LtA). Based on our analyses of the 
data collected using the original 18-item version (Lim, Morera, & Tchoshanov, 2009), we 
increased the number of items from 18 to 32 and revised some of them. To establish the validity 
of this enhanced version, we developed and administered an open-ended questionnaire, a 
classification exercise, and a multiple-choice mathematics test. 


Constructs Related to Impulsive-Analytic Disposition 

Impulsive disposition can be viewed from various perspectives. From a cognitive 
perspective, impulsive disposition is related to the Einstellung effect—the phenomenon of 
solving a given problem in a fixated manner even when a better approach exists (Luchins, 1942). 
Ben-Zeev and Star (2001) used the term spurious correlation to account for students’ 
association-based behavior: “when a student perceives a correlation between an irrelevant feature 
in a problem and the algorithm used for solving that problem and then proceeds to execute the 
algorithm when detecting the feature in a different problem” (p. 253). Einstellung effect, 
spurious correlation, and impulsive disposition emphasize different aspects of the same 
phenomenon: (a) Einstellung effect refers to a mental fixation, (b) spurious correlation refers to a 
feature-algorithm association, and (c) impulsive disposition refers to a cognitive tendency. 

From a problem-solving perspective, problem-solving disposition is related to metacognition. 
Actively monitoring one’s progress in relation to a goal is an indicator of analytic disposition. 
Impulsive disposition, on the other hand, is inferred when students “read, make a decision 
quickly, and pursue that direction come hell or high water” (Schoenfeld, 1992, p. 356). We 
consider impulsive disposition to be an externalization of certain beliefs such as “there is only 
one correct way to solve any mathematics problem—usually the rule the teacher has most 
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recently demonstrated to the class” (p. 359). 

From a psychological perspective, impulsive disposition can be viewed as a personality trait. 
Kagan et al. (1964) regard a child as (a) impulsive if the child responds to a question with an 
inaccurate answer but in a short response time, and (b) reflective if the child has an accurate 
answer but long response time. Nietfeld and Bosma (2003) found that college students’ 
impulsive-reflective style is consistent across three types of tasks: verbal, mathematical, and 
spatial. 

From a teaching-learning perspective, impulsive-analytic disposition can be viewed as ways 
of thinking (Harel, 2008) or habits of mind (Cuoco, Goldenberg, & Mark, 1996). In addition to 
helping students develop mathematical understanding, teachers can help students to develop 
desirable habits of mind such as being analytic in solving a problem instead of being impulsive 
by applying the first idea that comes to mind. Self-awareness is a crucial first step towards 
transforming habits of mind from undesirable to desirable. A way to create awareness is to have 
an instrument that can accurately assess one’s own impulsive-analytic disposition. 


Means to Assess Impulsive-Analytic Disposition 

A reliable way to investigate students’ problem solving behaviors is through task-based 
interviews (Clement, 2000; Goldin, 1998) and think-aloud protocols (Ericsson & Simon, 1993). 
Ways of thinking such as impulsive anticipation and analytic anticipation can be identified from 
the analysis of students’ responses to interview tasks (see Lim, 2006). Although well-suited for 
uncovering problem-solving disposition in individual students, this mode of data collection is not 
practical for large-scale assessment of students’ problem-solving disposition. 

Well-designed mathematical problems can be an effective and efficient means to assess 
impulsive-analytic disposition. Frederick (2005) designed a three-item test for assessing 
cognitive reflection. One of the items is: “A bat and a ball cost $1.10 in total. The bat costs $1.00 
more than the ball. How much does the ball cost?” (p. 27). The most common wrong answer, 10 
cents, is considered impulsive. A reflective person, on the other hand, is likely to realize that the 
difference between $1.00 and 10 cents is not $1.00 but 90 cents. Other problems, such as 
missing-value problems involving non-proportional situations, can also be used to elicit 
impulsive behaviors. 

An efficient way to measure cognitive and psychological constructs is through the use of 
questionnaire. For example, the Need for Cognition Scale (NfCS; Cacioppo & Petty, 1982) 
measures one’s desire to engage in a complex thought. In the context of mathematics problem 
solving, Lim et al. (2009) developed the LtA survey to measure problem-solving disposition 
along the impulsive-analytic dimension. The LtA survey consists of six-point Likert items where 
participants are asked to indicate how likely they are to respond to a given mathematical problem 
in the described manner. The LtA survey, like most Likert-scale questionnaires, can be used for 
self-assessment. 

The purpose of our study was two-fold. First, we sought to improve internal consistency 
reliability by lengthening the LtA survey to a 32-item measure. We then examined the validity of 
the LtA survey by determining whether the LtA scores were related to accuracy in classifying the 
LtA items and performance on a two-part mathematics test. 


Method 
Participants 


Wiest, L. R., & Lamberg, T. (Eds.). (2011). Proceedings of the 33rd Annual Meeting of the North 
American Chapter of the International Group for the Psychology of Mathematics Education. 
Reno, NV: University of Nevada, Reno. 


Articles published in the Proceedings are copyrighted by the authors. 


PME-NA 2011 Proceedings 286 


There were three groups of participants: (a) 27 in-service teachers and 10 pre-service 
teachers in a program for improving mathematics and science education in El Paso; (b) two 
mathematics classes for pre-service EC-8 (Early Childhood to Grade 8) teachers with 33 and 22 
students respectively; and (c) one class of 27 pre-service EC-8 teachers. Because this round of 
data collection was designed for testing and piloting the instruments, a convenience sample of in- 
service teachers was used. Data collection involving the first group was integrated into a 2.5- 
hour lesson on problem-solving disposition. The participants first took the LtA survey and an 
open-ended questionnaire. They then experienced their own problem-solving disposition via a 
three-problem activity involving clickers which are remote units for a personal response system 
that records student responses. They were introduced to impulsive and analytic dispositions via a 
PowerPoint Presentation and were then asked to classify each of the 32 LtA items based on 
whether they considered the act described in the item analytic or impulsive. The activities for the 
second group of participants differed slightly from that for the first group in that the three- 
problem clicker activity was excluded because of shorter class time. The third group of 
participants took the LtA survey and the two-part math test. 

Instruments 

Likelihood-to-Act survey. The current version of the LtA survey has four categories: algebra, 
fraction, word problem, and non-mathematically-specific description. In each category there are 
four impulsive items and four analytic items. A pair of items for each category is presented 
below. 

aA3 (x—7)(x — 4) = 0. When asked to solve for x, how likely are you to study the equation 
and predict the solution? [analytic, algebra] 
iA3 (x —5)(x — 8) =0. When asked to solve for x, how likely are you to multiply out the 
terms (i.e., FOIL) and then solve x” — 13x + 40 = 0 using the quadratic formula? 
[impulsive, algebra] 
aFl 24+U4 Le + = 
expression without using a calculator, how likely are you to begin by studying the 
fractions to see if you can predict the answer? [analytic, fraction] 
iFl 3+i4 2° + = 
expression without using a calculator, how likely are you to begin by finding the 
common denominator? [impulsive, fraction] 
aW3 Paula is cycling from home to school. At 8 o'clock she has already cycled 2.4 miles. 
When asked to find her rate of cycling, how likely are you to analyze the problem 
situation instead of dividing 2.4 by 8? [analytic, word problem] 
iW3 Jimmy is walking from home to school. At 7 o'clock he has already walked 1.4 km. 
When asked to find his rate of walking, how likely are you to use the d = rt or r= d/t 
relationship and obtain 0.2 km/hour? [impulsive, word problem] 
aG1 In solving a problem in mathematics, how likely are you to interpret and understand 
the problem thoroughly before deciding what to do? [analytic, general] 
iG1_ In solving a problem in mathematics, how likely are you to use the first idea that 
comes to mind? [impulsive, general] 
Three measures can be derived from the LtA survey: (a) the analytic subscale is based on the 16 
analytic LtA items, (b) the impulsive subscale is based on the 16 impulsive items, and (c) the 
analytic-impulsive difference is computed based on the difference between the analytic score and 
the impulsive score for each pair of items. 

Open-ended Questionnaire. In this questionnaire, six open-ended questions were posed to 

uncover participants’ initial approaches for solving selected impulsive items in the LtA survey. 


+ =. When asked to find the answer for the above arithmetic 


+ =. When asked to find the answer for the above arithmetic 
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The following question, for example, is associated with Item iA3: “What are the first few actions 
that you would take when asked to solve (x — 5)(x — 8) = 0 for x?” Two versions were created to 
cover the 12 non-general impulsive items (1.e., iA1-iA4, iF 1-iF4, and iW1-iW4). The first author 
and two first-year graduate students coded all the responses in one version. Another team of a 
full-time research assistant and a final-year doctoral student coded all the responses in the second 
version. Members from both teams met to analyze the responses in a training set and to discuss 
general principles for analyzing. The inter-rater reliabilities for the two teams were 0.91 and 0.97 
respectively. For each written response, two codes were assigned: (a) disposition code using a 
five-point scale to indicate whether the response has a strong or weak indication of analytic (A+ 
or A-) or impulsive (I+, I-) or is inconclusive (U); and (b) correctness code (1 = correct or no 
indication of misunderstanding, and 0 = presence of misconceptions or non-trivial errors). The 
analysis of students’ written responses was reported in the 2010 PMENA conference (Lim & 
Mendoza, 2010). 

Classification test. The classification test was designed to assess the accuracy in determining 
whether the act described in each LtA item is analytic or impulsive. For each of the 32 LtA 
items, participants were asked to label the item as analytic or impulsive, and then to rate whether 
they were confident or not confident in their answer. 

Belief survey. On a 10-point scale (1 = Impulsive; 10 = Analytic), participants were asked 
which point on the scale best describes their own problem-solving and that of their peers, high- 
school (HS) math students, HS science students, HS math teachers, and HS science teachers. On 
a five-point scale (1 = strongly disagree; 5 = strongly agree), participants were asked whether 
they agree with statements about their problem-solving disposition, the lesson/presentation on 
impulsive-analytic disposition, and the U.S. mathematics education with regards to impulsive 
disposition. Listed below are representatives of the statements in the survey. 

¢ Iconsider myself more impulsive than analytic. 

* Today’s lesson has helped me appreciate the importance of differentiating between impulsive 
disposition and analytic disposition. 

¢ U.S. high school mathematics curricula promote impulsive disposition. 

Two-part mathematics test. The test was divided into two parts to test the effect of warning 
about trickiness of problems on performance. The test items in both parts are structurally 
equivalent. Each part has 12 problems: 6 involving non-proportional situations and 6 involving 
fractions, ratios, or percents. The problems were designed such that students with an impulsive 
disposition would be more likely to choose a wrong answer choice. Below are two test items: 

¢ When a candle has burned 23 mm of its original length, its height is 82 mm. 
What is the candle’s height when it has burned 46 mm of its original length? 
(a) 41 mm (b) 59 mm (c) 164 mm (d) None of the above 
¢ Benito needs to increase the money he has now by 20% so that he can buy a $540 
laptop. How much more money does he need? 
(a) $90 (b) $108 (c) $432 (d) $2700. 
After completing the first part and before beginning the second part, an example (see Figure 1) 
was presented to caution participants that “some of the items in this test may be considered 
‘tricky’ for some students.” 
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Consider the following problem: 


John bought a new car this year. If the car depreciates by 20% each year, 
the car will have depreciated by in two years. 
(a) 30% (b) 36% (c) 40% (d) 64% 


Many students choose “c” because 20% plus 20% is 40%, but the correct answer is “b”. 

The actual depreciation in dollar amount in the second year is less than that in the first year 
because the cost of the car after 1 year 1s less. For example, suppose the new car costs $10,000. 
After 1 year, the car’s value drops by $2000 (20% of $10000), and it is now worth $8000. 

In the second year, its value drops by $1600 (20% of $8000). So after 2 years, its value drops 
by $3600, which is 36% of $10000. 


Figure 1: An example to warn students to be cautious prior to working on Part 2 


Results and Discussion 

Reliability of the Various Measures 

The internal consistency reliability estimates for the various measures in each instrument 
were calculated using Cronbach’s alpha. Table 1 presents for each measure the number of items 
in the measure, the mean value, the standard deviation, the a-value, and the 95% confidence 
interval. The reliability values for most of the measures are relatively high, except for the Coded- 
correctness score from the open-ended questionnaire and Part 1 of the mathematics test. The 
reliabilities for the analytic and impulsive subscales, 0.81 and 0.74 respectively, were greater for 
this version of the LtA survey than those in the previous version, 0.63 for the 7-item analytic 
subscale and 0.64 for the 7-item impulsive subscale (Lim et al., 2009). 

Table 1: Descriptive and Reliability Measures for the Various Instruments 
No. of Mean Standard Cronbach 95% Conf. 


Items Deviation OL Interval for o 

LtA Survey (NV = 119) 

Analytic Subscale 16 3.96 0.71 0.81 0.76 , 0.86 

Impulsive Subscale 16 4.45 0.63 0.74 0.67 , 0.81 

Analytic-Impulsive Difference 16 -0.49 0.83 0.77 O27, O83 
Open-ended Questionnaire (N = 92) 

Coded-disposition Score 6 1.86 0.31 0.71 0.61 , 0.79 

Coded-correctness Score 6 0.77 0.26 0.29 0.38 , 0.49 
LtA Classification Test (V = 92) 

Classification-accuracy Score 32 0.59 OZ 0.76 0.69 , 0.83 

Classification-confidence Score 3p 0.75 0.09 0.86 0.81 , 0.90 
Belief Questionnaire (N = 92) 

Self-disposition Belief* 2 2.88 0.38 0.84 WEES eee 

Lesson-on-disposition Opinion? 4 4.43 0.05 0.93 O90 e095 
Two-part Math Test (NV = 27) 

Math-Part1 Score*® 10 0.28 0.22 0.57 0.28 , 0.76 

Math-Part2 Score 12 (32 0.22 0.67 0.46 , 0.83 


“The 10-point scale item was transformed into a 5-point scale. The 5-point item was reverse coded. 
All the four items about the lesson used a 5-point scale. 
“Two items had zero variance (all 27 students got them wrong) and were removed from the scale. 
A mean value of -0.49 for analytic-impulsive difference indicates that the participants in this 
study chose lower values for analytic items than for impulsive items in the LtA survey. A mean 
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value of 1.86 for the coded-disposition score on a 5-point scale (1 = impulsive; 5 = analytic) also 
indicates that the 92 participants were generally more impulsive than analytic in their open- 
ended responses. A mean value of 2.88 for the Self-disposition Belief score on a five-point scale, 
on the other hand, indicates that the participants view themselves as more analytic than 
impulsive. Mean values of 0.23 (12 items; 0.28 is Table 1 was based on 10 items) and 0.32 (12 
items) for the two parts of the math test indicate that the 27 pre-service teachers on average 
responded correctly to 23% and 32% of the 12 items. These findings suggest that participants are 
generally more impulsive than analytic although they might view themselves as more analytic 
than impulsive. 

A mean value of 4.43 on a 5-point scale indicates that the participants have a favorable 
opinion about the lesson on impulsive and analytic dispositions. A mean value of 0.59 for the 
classification-accuracy score indicates that the 92 participants (Groups | and 2) have a 59% 
accuracy in classifying the 32 LtA items. The 27 participants (Group 3) showed a 9% gain from 
Part 1 to Part 2, suggesting that students performed better when they were warned to be cautious. 
Correlations among the Various Measures 

The correlation between the analytic subscale and impulsive subscale was -0.014, not 
statistically different from zero. In other words, the analytic and impulsive subscales were 
independent of one another. Interestingly, the difference score between the two subscale scores 
for each pair of items had a relatively high reliability of 0.77 (see Table 1). 

Table 2 presents the correlation between the measures from the LtA survey and other 
measures that might be related to impulsive-analytic disposition. The disposition scores derived 
from coding of the six open-ended responses were positively correlated (7 = 0.373) to the 
analytic subscale and negatively correlated (7 = -.0488) to the impulsive subscale. These 
significant correlations were probably due to the similarity between the items in the open-ended 
questionnaire and the LtA items. In addition, their initial exposure to the LtA survey first might 
have influenced their subsequent written responses. 

Table 2: Correlations between LtA Subscales and Other Measures 


Analytic Impulsive 

Subscale Subscale 
Analytic Subscale 1 
Impulsive Subscale -0.014 1 
Analytic-Impulsive Difference O745** -0.678** 
Coded-disposition Score 0:373** -0.488** 
Coded-correctness Score 0.256* -0.205* 
Classification-accuracy Score 0.107 -0.286** 
Classification-confidence Score 0.183 -0.048 
Self-disposition Belief Score 0.418** “0:27 3" 
Lesson-on-disposition Score 0.011 -0.041 
Math-Part1 Score* 0.075 -0.205 
Math-Part2 Score Q95"* -0.140 


"p< .05, p<.01. 
Participant accuracy in classifying items was negatively correlated to impulsive subscale 
(r = -0.286) but not correlated to analytic subscale. This finding suggests that students who have 
an impulsive disposition tended to make more mistakes in classifying items. Participant 
confidence in their classification was not correlated to either of the LtA subscales. 
Participant self-reported disposition score was correlated to both the LtA subscales. The 
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lesson-on-disposition score was not correlated to either of the LtA subscales. 

Participant performance in Part 1 of the mathematics test, prior to the warning, was not 
correlated to the LtA subscales. On the other hand, student performance in Part 2 was correlated 
significantly to the analytic subscale but not to the impulsive subscale. This suggests that the 
warning about problems being tricky had an impact on students with an analytic disposition but 
not those with an impulsive disposition. 

In summary, the two LtA subscales were reliable and uncorrelated. Other measures in the 
study were reliable and were appropriately associated with the LtA constructs. 

Concurrent Validity 

In addition to the above analyses, we were also interested in determining whether the LtA 
subscales could be used to predict the following variables: (a) self-reported GPA, (b) 
classification-accuracy score, (c) lesson-on-disposition score, (d) Math-Part1 score, and (e) 
Math-Part2 score. To perform these analyses, a hierarchical multiple regression was performed. 
In the first step of the multiple regression, variables representing participant characteristics 
(years of teaching experience, sex, grade band and subject area currently teaching or planning to 
teach) were entered into the regression model. In the second step, we typically entered the two 
LtA subscales (except for predicting Math-Part2 score). None of the variables mentioned above 
could statistically explain variability in self-reported GPA, the lesson-on-disposition score, and 
the Math-Part1 score. In other words, the LtA scores could not predict these three variables, but 
the LtA scores did predict classification-accuracy score and the Math-Part2 score. 

Predicting Classification-accuracy Score. The variables entered in the first step of the 
regression model explained 3.3% of variability in the classification-accuracy score. None of 
these predictors was statistically significant. When the LtA subscales were added, the fraction of 
variance explained, R’, increased by 10.3% to 13.6%. The impulsive subscale (f = -.316, 

p = 0.011) accounted for a significant portion of the variance. In other words, increased 
impulsivity was associated with worse item classification. 

Predicting Math-Part2 score. This regression consisted of four steps instead of two. In the 
first step, participant sex and indicator variables representing grade band were entered and 
accounted for 3.5% of variability in the Math-Part2 scores. None of these predictors was 
statistically significant. In the second step, Math-Part1 score was entered and accounted for an 
additional 27.8% of variability. Increased Math-Partl scores were associated with increased 
Math-Part2 scores ($ = .556, p = .010). In the third step, the LtA subscale scores were entered 
and accounted for an additional 37.8% of variability. Increased analytic disposition (f = .652, p = 
.000) was associated with increased Math-Part2 scores. In the final step, values representing the 
interaction between Math-Part! and the LtA subscales were entered into the regression model. 
These interactions explained an additional 13.1% of variability. The interaction between Math- 
Partl and analytic disposition was statistically significant (B = 1.75, p = .004) in that individuals 
who had both high Math-Part1 scores and high analytic scores had higher Math-Part2 scores. 


Concluding Remarks 
The results obtained in this research phase indicate that a measure of problem-solving 
disposition along the analytic-impulsive dimension could be created using the LtA survey. The 
LtA scales demonstrated adequate internal consistency reliability. In addition, we found that the 
analytic and impulsive subscales were independent of one another, replicating the result of an 
earlier study in the behavioral decision theory on types of decision making styles (Morera et al., 
2006). 
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In this round of data collection, participants were also asked to complete a series of other 
measures. The analytic and impulsive LtA scores were found to be correlated to certain scores 
derived from the open-ended questionnaire, the belief survey, and the two-part mathematics test. 
In particular, the impulsive score was found to be predictive of correctness in classifying the LtA 
items, and the analytic score was predictive of how well a participant would perform in Part 2 of 
the math test after the warning, while controlling for the effects of Math-Part 1 scores. In 
summary, these findings represent a first step in demonstrating the utility of the LtA measures. 

The data we obtained were used to refine the LtA survey (see Lim & Mendoza, 2010) and the 
two-part mathematics test. The revised versions of the two instruments have been administered 
to more than 450 pre-service teachers. The Barratt Impulsiveness Scale Version 11 questionnaire 
(Patton et al., 1995), the NfCS questionnaire (Cacioppo & Petty, 1992), and the Cognitive 
Reflection Test (Frederick, 2005) were administered concurrently to determine how well the LtA 
survey correlates with these three instruments and how well the LtA scores predict students’ 
performance in the two-part mathematics test. 
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